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problems with state constraints where the state equation is a differential equation with delays. 
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1 Introduction 

The main purpose of this paper is to prove the existence of optimal feedback strategies for a 
class of optimal control problems of deterministic delay equations arising in economic models. 

The paper represents the natural continuation of [21] where a class of optimal control prob- 
lems with state constraints where the state equation is a differential equation with delays is 
studied. This class includes some problems arising in economics, in particular the so-called 
models with time to build. In [21] the problem is embedded in a suitable Hilbert space H and 
the associated Hamilton-Jacobi-Bellman (HJB) equation is studied. Therein the main result is 
concerned the regularity of solutions to such a HJB equation. More precisely it is shown that 
the value function has continuous classical derivative in the direction of the "present". This 
allows to define a feedback strategy in classical sense. 

In the present paper we start from this result and we exploit it to prove: 

• the existence of optimal feedback strategies through a Verification Theorem; 

• the existence of e-optimal strategies for a wider family of problems through approximation 
results. 

The class of optimal control problems is the following: given a control c > the state x 
satisfies the following delay equation 

x'{t) = rxit) + /o aiOx{t + C)d^) - c(t), 

x(0) = 7?o, x{s) = r/i(s), s G [-T,0), 

with state constraint x{-) > 0. The objective is to maximize the functional 

r+oo 

J(r/; c(-)) := / e"^* [UMt)) + ^72(x(t))] dt, p > 0, 
Jo 

over the set of the admissible controls c. 
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When the feedback strategy effectively exists and is admissible we prove (Theorem I3.6P that 
it must be optimal for the problem: this is not trivial since we do not have the full gradient of 
the value function and so we need to use a verification theorem for viscosity solution which is 
new in this context. Indeed a verification theorem in the framework of viscosity solution is given 
in the finite dimensional case in [31]. Adapting the technique of proof to our case is difficult 
due to the infinite dimensional nature of our problem and to a mistake in the key Lemma 5.2, 
Chapter 5 of [31] that we discovered here and that is pointed out in Remark 13.51 We then give 
(Proposition I3.1ip sufficient conditions under which the formal optimal feedback exists and is 
admissible. 

Since our setting (where we prove the Verification Theorem and the existence of optimal 
feedback strategies) do not cover the case of pointwise delay (see [21], Remark 4.8) which is used 
in the previously quoted applications, we go further showing three approximation results that 
allow to apply our main theorem to obtain e-optimal strategies for a wider class of problems 
including the case of pointwise delay (Propositions 14.31 14.61 14.8P . 

The plan of the paper is as follows. Section [2] is devoted to recall the problem set in [21] 
and the results contained therein. Section 3 contains the first main result, i.e. the Verification 
Theorem l3.61 Then, in Section 3.1 we give sufficient conditions under which the hypothesis of the 
verification theorem is satisfied. Section S] closes the paper with the announced approximation 
results. 

2 The optimal control problem 

In this section we give the setup of the optimal control problem and recall, for the reader's 
convenience, the main results of [21]. We will use the notations 

LIt ■= L^{[-T,0];R), and W]_'^ := W^'^{[-T,0];R). 

We will denote by H the Hilbert space 

if := M X 

endowed with the inner product 

(•, •) = (•, ■)m. + (•, Ol^j,) 

and the norm 

II l|2 _ I |2 I II ||2 

II • II - I • Ir + II • IIl2 

We will denote by r] := (r/o, Vii')) generic element of this space. For convenience we set also 

H+ := (0,+oo) X LIt, H++ := (0, +oo) x {7?i(-) € L^^ I m(-) > a.e.}. 

Remark 2.1. Economic motivations we are mainly interested in (see [Il[2l[27]) require to study 
the optimal control problem with the initial condition in However, the set is not 

convenient to work with, since its interior with respect to the || • ||-norm is empty. That is why 
we enlarge the problem and allow the initial state belonging to the class Hj^. ■ 
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For r] G Hj^, we consider an optimal control of the following differential delay equation: 

fx'(i) = rx{t) + fo (x(i), + m) - c{t), 

[x{0) = r/o, x{s) = 7?i(s), s G [-r,0), 

with state constraint x(-) > and control constraint c(-) > 0. We set up the following assump- 
tions on the functions a, fo- 

Hypothesis 2.2. 

• a(-) G W^'!^ is such that a(-) > and a{—T) = 0; 

• fo '■ [0, oo) X M — M is jointly concave, nondecreasing with respect to the second variable, 
Lipschitz continuous with Lipschitz constant C/g, and 

/o(0,y)>0, Vy>0. (2) 



Remark 2.3. In the papers [H [2l [27] the pointwise delay is used. We cannot treat exactly 
this case for technical reason that are explained in Remark 14.71 below. However we have the 
freedom of choosing the function o in a wide class and this allows to take account of various 
economic phenomena. Moreover we can approximate the pointwise delay with suitable sequence 
of functions {an} getting convergence of the value function and constructing e-optimal strategies 
(see Subsections 14.21 and 14. 3p . ■ 

We say that a function x : [—T,oo) — > M"*" is a solution to equation ([T|) if x{t) = r]i{t) for 
t G [-T, 0) and 

x(t)=r/o + j rx{s)ds + j fo(^x{s),j a{^_)x{s + ^)d^^ ds - j c{s)ds, t>0. 

Theorem 2.4. For any given rj G c(-) G L^^^^([0, +00); equation ([1]) admits a unique 

solution that is absolutely continuous on [0, +00) . 

Proof. See ED. □ 



We denote by x(-;r;, c(-)) the unique solution of ([T|) with initial point 77 G and under 
the control c(-). We emphasize that this solution actually satisfies pointwise only the integral 
equation associated with ([1]); it satisfies ([T|) in differential form only for almost every t G [0, +00). 

For r] G Hj^ we define the class of the admissible controls starting from 77 as 
C(r/) := {c(-) G LL([0,+oo);M+) | x(-;r?,c(-)) > 0}. 
Setting x{-) := x(-, ;r/, c(-)), the problem consists in maximizing the functional 

Jiri; c(-)) := / e^P' [UMt)) + C/2(x(t))] dt, p > 0, 







over the set of the admissible strategies. 

The following will be standing assumptions on the utility functions Ui, U2, holding through- 
out the whole paper. 
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Hypothesis 2.5. 

(i) Ui £ C([0,+cx));M) nC2((0,+oo);]R), U[ > 0, U[{0+) = +00, < and Ui is bounded. 

(ii) U2 G C((0, +00); M) is increasing, concave, bounded from above. Moreover 



e~ 



P'U2 (e-^/o*) dt> -00. (3) 



Since Ui, U2 are bounded from above, the previous functional is well-defined for any rj € 
and c(-) G C{i]). We set 

Ui := lim Ui{s), U2 := lim U2{s). 

s— »+oo s—*+oo 

We refer to [21] for comments on the assumptions above. 
For T] G the value function of our problem is defined by 

Virj):= sup J(r?,c(-)), (4) 

c{.)eC(ry) 

with the convention sup0 = — 00. The domain of the value function is the set 

V{V) := {v^H+\ V{ri) > -00}. 
Due to the assumptions on Ui, U2 we directly get that V < j;{Ui + ^^2)- 

2.1 Preliminary results 

The proof of the following qualitative results on the value function can be found in |21j . 

Lemma 2.6 (Comparison). Let rj G and let c(-) G L[q^([0, +00); M"^). Let t > 0, 6e an 
absolutely continuous function satisfying almost everywhere the differential inequality 

ix'it) < rx(t) + /o (x{t), j% a{Ox{t + ^)d^) - c{t), 
[x{0) < rjo, x{s) < r/i(s), for a.e. s G [-T,0). 

Then x{-) < X {■■,!], c{-)). □ 
Proposition 2.7. VKe have 

C = {r/ G I G C(77)}. 

T/ie set 'E'(V^) is convex and the value function V is concave on ViV). □ 
Proposition 2.8. We have the following statements: 
1- V{r]) <-p{Ui + U2) for any r] H+. 

2. lim^,^+oo^(??o,m(-)) = -p{Ui + U2), for all G L^^. 

3. V is strictly increasing with respect to the first variable. 

□ 
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2.2 The delay problem rephrased in infinite dimension 

Our aim is to apply the dynamic programming technique in order to solve the control problem 
described in the previous section. However, this approach requires a markovian setting. That 
is why we will reformulate the problem as an infinite-dimensional control problem. Let n = 
(1,0) G and let us consider, for 77 € and c(-) e L^([0, +00); the following evolution 
equation in the space H: 

(x'{t) = AXit)+F{Xit))-c{t)h, 

In the equation above: 

• A : T>{A) C H — > H is an unbounded operator defined by A(r/o, '■= 

V{A) := {i]eH\ mi-) G W'j^, r/i(0) = %}; 

• F : H — > H is a Lipschitz continuous map defined by 

where /(r/o, := /o (voJ^t "-(OmiOd^y 

It is well known that A is the infinitesimal generator of a strongly continuous semigroup {S{t))t>o 
on H; its explicit expression is given by 

S{t)im,m{-)) = {voe'',I[-T,o]{t + -) Vi{t + ■) + I[o,+oo){t + ■) %e^^*+-^) ; 
2.2.1 Mild solutions of the state equation 

Here we give a definition of the mild solution to ([5]), state the existence and uniqueness of 
such a solution and the equivalence between the one dimensional delay problem and the infinite 
dimensional one. We refer to [21] for the proofs. 



Definition 2.9. A mild solution of ([5]) is a function X G C{[0,+oo); H) which satisfies the 
integral equation 



X{t)=S{t)r]+ [ S{t-T)F{X{T))dT + [ c{T)S{t-T)ndT. 
Jo Jo 



(6) 



Theorem 2.10. For any t] G H, there exists a unique mild solution of □ 

We denote by X{-;i],c{-)) = (Xo(-; c(-)), Xi (•; r/, c(-))) the unique solution to ([5]) for the 
initial state r] £ H and under the control c(-) S L^([0, +00); M^). The following equivalence 
result justifies our approach. 

Proposition 2.11. Let rj G H^, c(-) G C(r/) and let x(-), X{-) be respectively the unique solution 
to ([2P and the unique mild solution to (0j starting from r] and under the control c(-). Then, for 
any t > 0, we have the equality in H 

X{t) = (x(t),x(t + 05ehT,o])- 

□ 
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2.2.2 Regularity of the value function 

Here we state the regularity properties of the value function. We refer to [21] for the proofs. 
We recall that the generator A of the semigroup {S{t))t>o has bounded inverse in H given 

by 

A-^ (%,r/i) (s) ={j,^- J'^nim) , s G [-r,0]. 

It is well known that A~^ is compact in H. It is also clear that A^^ is an isomorphism of H 
onto T>{A) endowed with the graph norm. 
We define the || • ||„i-norm on H by 

||r7||_i := ll^d""^?]!!. 

Proposition 2.12. The set ^^(V) is open in the space {H, \\ ■ and the value function is 
continuous with respect to \\ ■ ||_i onV{V). Moreover 

{r,n)cV{V), i]n^r,eV{V) =^ V {r,n) ^ V {r]) . (7) 

□ 

Therefore, we can apply the following result to the value function. 

Proposition 2.13. Let v : 'D(V) ^ M. be a concave function continuous with respect to \\ ■ 
Then 

1. V = u o A~^ , where u : O C H ^ is a concave \\ ■ \\- continuous function. 

2. D^vij]) C V{A*), for any r] £ V{V). 

3. D~^u{A''^r]) = A*D~^v{r]), for any rj € 'DiV). In particular, since A* is injective, v is 
difjerentiable at rj if and only if u is differentiable at A~^r]. 

4- If C ^ D*v{r]), then there exists a sequence rjn ^ rj such that there exist Vv{rjn), ^v{r]n) — 
C and A*Vv{r]n) A*C,. 

□ 

The HJB equation associated to our optimization problem is 

pv{in) = {r,,A*Vv{r]))+f{i^)v^,{r,) + U2{vo)+n{v^M)^ (8) 
where 7i is the Legendre transform of Ui, i.e. 

7^(Co) :=sup([/i(c)-Coc), Co>0. 

c>0 

Due to Hvphothesis I2.5l -(i) and to Corollary 26.4.1 of [30], we have that ?i is strictly convex on 
(0,+oo). Notice that, thanks to Proposition 12.81 (3). 

Kyil) ■■= {Co G M I (Co,Ci(-)) G D+Viv)} C (0,oo) 
for any rj € D{V), i.e. where 7i is defined. 
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We can study this equation following the viscosity approach. In order to do that, we have 
to define a suitable set of regular test functions. This is the set 

r := {(^ G C\H) \ G V{A*), rin ^ V ^ A*V^{rin) ^ A*\/^{r,)]. (9) 

Let us define, for c > 0, the operator on r by 

[C'ifM := -pviv) + (r?,^*Vv9(r/)) + f{v)^.^oiv) " Cifr^oir])- 

Definition 2.14. (i) A continuous function v : ViV) ^ M is called a viscosity subsolution of 
([8]) on T>{V) if for any if £ t and any rfM € 1^{V) such that v — ip has a || • ||-local maximum at 
rjM we have 

pv{mi) < (??A/,^*V(/7(r?A/)) + f{VAl)^rio{^M) + C^2(??o) + W((/5,,o (^A/))- 

(ii) A continuous function w : ^^(V) ^ M is called a viscosity supersolution of ([8]) on if 
for any if £ t and any r]m € ^^(^) such that v — ip has a || • ||-local minimum at rjm we have 

(iii) A continuous function v : T^iV) ^ M is called a viscosity supersolution of ^ on T>(y) 
if it is both a viscosity sub and supersolution. 

Theorem 2.15. The value function V is a viscosity solution of on 'D(V). □ 

Actually the concave || • ||_i-continuous viscosity solutions of ^ (so that in particular the 
value function V) are differentiable along the direction n = (1,0). This is stated in the next 
result: we refer to [21] for the proof. 

Theorem 2.16. Let v be a concave \\ ■ \\-i-continuous viscosity solution of ^ on ViV). Then v 
is differentiable along the direction h = (1, 0) at any point rj € 1^(1^) and the function rj i— > VriQ^rj) 
is continuous on □ 



3 Verification theorem and optimal feedback strategies 

Here we prove a Verification Theorem yielding optimal strategies for the problem. We start with 
the following definition. 

Definition 3.1. Let rj € ^{V). An admissible control c*(-) G C{t]) is said to be optimal 
for the initial state r/ if J{r];c*{-)) = V{rj). In this case the corresponding state trajectory 
x*{-) := x{-;r],c* {■)) is said to be an optimal trajectory and the couple {x* {■) , c* {■)) is said an 
optimal couple. □ 

Thanks to the regularity result of the previous section we can define, at least formally, the 
"candidate" optimal feedback map on T>{V), which is given by 

Civ) := argmax,>o {Ui{c) - cVr„{v)) , r, G V{V). (10) 

Note that this map is well-defined since V is concave and, by Proposition [221 strictly increasing, 
so that we have 14yo(^) ^ (0, +oo) for all r] G 'DCV). Existence and uniqueness of the argmax 
follow from the assumptions on Ui. Moreover, since is continuous on DlV), also C is 
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continuous on 'D(V). The closed-loop delay state equation associated with this map is, for 



'x'it) = rx{t) + /o (x{t),J% a{Ox{t + C)d^) - C {{x{t),x{t + Ol5e[-T,o])) , 
_x(0) = 7?o, x{s) = r/i(s), s G [-T,0). 



(11) 

Now we want to prove a Verification Theorem: if the closed loop equation (jlip has a strictly 
positive solution (so that we must have (a;*(t), + ^)|^g[„j'o]) € T^lV) and the term 

C (^{x* (t) , X* {t + ^)|^g[_7io])) is well-defined for every t > 0), then the feedback strategy 

c*{t):=C{{x*{t),x*{t + Okel~T,o])) (12) 

is optimal. Notice that, by definition of c*(-), if x*{-) is a strictly positive solution of (jlip . then 
c*(-) is admissible and, setting X*{t) := X {t; t] , c* {■)) , we have 

x*it) = {x*it),x*{t + Oke[-m) ^'Div), vt>o. 

In order to prove a Verification Theorem, formally we need to integrate the function 

t^j^ie-^'ViX^t))]. (13) 

Thus we need something like a Fundamental Theorem of Calculus relating the function and the 
integral of its "derivative". Since we do not require the intial datum r] belonging to 'D(A) and 
the operator A works as a shift operator on the infinite-dimensional component, we do not have 
the condition X*{t) G T^iA) for almost every t > giving a regularity for the function 

e-f"V{X*{t)) 

sufficient to apply the Fundamental Theorem of Calculus (see [28], Theorems 5.4, 5.5, Chapter 
6). Therefore we can suppose only that the function (jlSh is continuous and we should try 
to apply a generalized Fundamental Theorem of Calculus in inequality form. There is such a 
result in (31], Lemma 5.2, Chapter 5. Unfortunately such result is not true as it is stated (see 
Remark 13.51 for a counterexample), so we have to refer to other results based on the theory first 
formulated by Dini and Lebesgue. We refer to [TOl [251 129] sketching the ideas we need. If is a 
continuous function on some interval [a, f3] C K, the right Dini derivatives of g are defined by 

n+ m r g(t + h)- g{t) . ^ g{t + h) - g{t) 

D^git) = Imisup ; , Dj^git) = limmt ; , t G a,p), 

/40 h HO h 

and the left Dini derivatives by 

I)-,(t)=limsup^^i±4^, I)_,(t)=liminf^^^±^?^^, tG(a,/3]. 

The key result is the following (see [10], Theorem 1.2, Chapter 4). 

Proposition 3.2. If g is a continuous real function on [a,f3], then the bounds of each Dini's 
derivative are equal to the bounds of the set of the difference quotients 



9{t)-9{s) 



t, s G [a, /3] 



□ 
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An immediate consequence of Proposition 13.21 above is the following. 



Proposition 3.3 (Monotonicity result). Let g G C([a, /3];M) be such that D^g[t) > for all 
t S [a, (3). Then g is nondecreasing on [a,f3]. □ 

Now we can give a simple lemma useful for proving the Verification Theorem. 

Lemma 3.4. Let g,^J, ^ C([0,+oo);M) such that 

D-g{t) > yt £ (0,+oo). (14) 

Then, for every < a < P < +oo, 

9{(3) - g{a) > / Kt)dt. (15) 

J a 

□ 

Proof. Since D-g{t) > fi{t) for every t € (0, +00), we have D-[g{t) — n{s)ds] > for 
every t E (0, +00). Thanks to Proposition 13.21 we have also D~^[g{t) — Jq ^'(s)(is] > for every 
t e [0,+oo). Therefore, due to Proposition 13.31 * 9{t) ~ Jq lJ'{s)ds is nondecreasing, getting 
the claim. □ 

Remark 3.5. Following |25|, we give some remarks on Lemma 13.41 

• The assumption that /i is continuous can be replaced assuming that // is a finite-valued 
(Lebesgue) measurable and integrable function (Theorem 9 of [25]); also condition (|14p 
can be weakened assuming that it holds out of a countable set (Section 5.b of |25j). 

• Condition (|14p can be weakened assuming that it holds almost everywhere adding the 
assumption D^g > —00 everywhere (Section 5.c of |25|). 



We cannot further weaken ()14p : if it is verified only almost everywhere without any further 
assumption on D^g, then (jl5p is no longer true. For example, if g = — / on [0, 1], where 
/ is the Cantor function and fi = 0, we have 

/i(t) = = g'{t) = D_g{t) for a.e. t G (0, 1]. 



Therefore, taking a = 0, /3 = 1, the left handside of (I15p is —1, while the right handside 
is 0. Indeed in this case D-g = —00 on the Cantor set. So Lemma 5.2, Chapter 5, of 
[3T] is not correct. Indeed the condition required therein is not sufficient to apply Fatou's 
Lemma in the proof. ■ 

Theorem 3.6 (Verification). Let 1] G and let x*{-) be a solution of (jlip such that x*{-) > 0; 
let c*(-) be the strategy defined by (jl2p . Then c*(-) is admissible and optimal for the problem. 

Proof. As said above the fact that c*(-) is admissible is a direct consequence of the assump- 
tion x*{-) > and of the definition of c*(-). 

Set X*{-) := X{-;r],c*{-)) and let s > 0. Let pi{s) G L^^ be such that 

{V,,{X*{s)),p,{s))eD+V{X*{s)) 
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and let 
so that 



¥.(0 := V{X*{s)) + {{V,,{X*{s)),pi{s)) , ( - X*{s)), ( G H, 



ip{X*is)) = V{X*is)), ^iC)>viC), C<^H. 
From Proposition 12.131 we know that 93 € r, so that 



_ ^e-P^'+^^V(X*(s + h)) -e-P'V(X*(s)) . ^e-P^'+''^ip{X*{s + h)) - e~P'ip{X*{s)) 

hm mi ; > hm mi 

hto h hio h 



{X*{s)) = e~P^ [ - pV{X*{s)) + {X{s),A* {V,,{X* {s)),p,{s))) 

+ f{X\s))V^,{X*{s))+c*{s)V^,{X*{s)) 

Due to the definition of c*(-) we get 



Um inf ■ 

/iTO 



-p{s+h)y(^X*{s + h)) 



-ps 



V{X*{s)) 



h 



+ e-P'[Ui{c%s)) + U2{X*,{s))] 



> e-P' 



pV{X\s)) + {X\s\A* {y^,(X\s)),vi{s))) 

+ j{x\sWr,,{x\s)) + n{x\s)) + u^{xi{s)) 

Due to the subsolution property of V we get 

e-''("+^V(X*(s + h)) - e~P'V{X\s)) 



hm inf ■ 



+ e-^^[C/i(c*(s)) + [/2(Xo*(5))] >0. 



The function s 1-^ e p^V{X*{s)) and the function s 1-^ e p^\U\(c*{s)) + \J2{Xl{s))\ are continu- 
ous; therefore we can apply Lemma [33] on [0, M], M > 0, getting 

e-P^'V{X*{M)) + / e~P'[Ui{c*{s)) + U2{X*^{s))]ds > Viv). 
Jo 

Since V,Ui,U2 are bounded from above, taking the limsup for M —f +00 we get by Fatou's 
Lemma 



+00 



e-PnU,ic*is)) + U2iX^is))]ds > Viv), 



which gives the claim. 



□ 



Remark 3.7. We have given in Theorem 13.61 a sufficient condition of optimality: indeed, we 
have proved that if the feedback map defines an admissible strategy then such a strategy is 
optimal. Of course, a natural question arising is whether, at least with a special choice of data, 
such a condition is also necessary for the optimality, i.e. if, given an optimal strategy, it can be 
written as feedback of the associated optimal state. From the viscosity point of view the answer 
to this question relies in requiring that the value function is a bilateral viscosity subsolution of 
([8]) along the optimal state trajectory, i.e. requiring that the value function satisfies the property 
of Definition 12. 14l -(i) also with the reverted inequality along this trajectory. 

Such a property of the value function is related to the so-called backward dynamic program- 
ming principle which is, in turn, related to the backward study of the state equation (see [8], 
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Chapter III, Section 2.3). Differently from the finite-dimensional case, this topic is not standard 
in infinite-dimension unless the operator A is the generator of a strongly continuous group, which 
is not our case. 

However, in our case we can use the delay original setting of the state equation to approach 
this topic. Then the problem reduces to find, at least for sufficient regular data, a backward 
continuation of the solution. This problem is faced, e.g., in [26], Chapter 2, Section 5. Unfor- 
tunately our equation does not fit the main assumption required therein, which in our setting 
basically corresponds to require that the function a(-), seen as measure, has an atom at — T. 
Investigation on this is left for future research. ■ 



3.1 The closed loop equation 

Up to now we did not make any further assumption on the functions a and U2 beyond Hypotheses 
12.21 and 12.51 in particular it could be U2 = 0. However without any further assumption we have 
no information on the behaviour of when we approach the boundary of T>{V) and therefore 
we are not able to say anything about the existence of solutions of the closed loop equation 
and whether they satisfy or not the state constraint. So basically we cannot say whether the 
hypothesis of Theorem 13.61 is satisfied or not. In order to give sufficient conditions for that, we 
need to do some further assumptions. 

Hypothesis 3.8. We will make use of the following assumptions 

^0 

(i) U2 is not integrable at 0+, (ii) / a{C)d^ > 0, Ve > 0. (16) 



Also we need the following Lemma; we refer to |21| for the proof. 

Lemma 3.9. Let X{-), X{-) be the mild solutions to starting respectively from r],f} & H and 
both under the null control. Then there exists a constant C > such that 

||X(t)-X(t)||_i <C||r/-7?||_i, yte[0,T]. 

In particular 

\Xo{t)-Xo{t)\<rC\\r]-fj\\^i, Vt G [0,r]. 

□ 

Lemma 3.10. 

1. The following holds 

d\\.\\V{V) = d\\.\\_^V{V). 

Thanks to the previous equality we write without ambiguity dT>{V) for denoting the bound- 
ary ofT>{V) referred to \\ ■ \\ or \\ ■ indifferentely. 

2. Suppose that p^ -/^ij holds; then 

lim K„(r/) = +00, Vfy G dViV), 

where the limit is taken with respect to \\ ■ \\. 
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Proof. We work with the original one-dimensional state equation with delay. 

1. First of all note that, thanks to Proposition 12.81 and Proposition 12.12] the set T>{V) has 
the following structure 

W= U H\+^)^{Vi}), (17) 

where, for 7]i G L'^_']^, we set r]^^ = infjryo > | ^ ^(^)}- For any 7] £ H set 

x''^{-) := x{-; T], 0) and consider the function g : H R. defined by 

te[o,T] 

Thanks to Lemma 13.91 this function is continuous (with respect to both the norms || • || and 
II • ll-i), so we have the following representation of 'DiV) in terms of g: 

V{V) = {g> 0}. 

Lemma 12.61 shows that g is increasing with respect to the first variable. Actually g is strictly 
increasing with respect to the first variable. Let us show this fact. Let r/i G LPij- and take 
r/o,r?o e M such that % > Define y(-) := (770, 0)) := r/i(-)), 0) and let 

z{-),z{-) be respectively the solutions on [0,T] of the differential problems without delay 

z'{t) = rzit) + /o (^(t), J% a{0<t + OdC) , 

z'it) = rzit) + /o [m, /°T <0<t + Od^) , 

Then we have, on the interval [0, T], z(-) = x{-) and, by comparison criterion, y(-) > z(-); 
moreover we can apply the classic Cauchy-Lipschitz Theorem for ODEs getting uniqueness for 
the solutions of the above ODEs, which yields z{-) > z{-) on [0,r]. Thus y(-) > x{-) on [0,T], 
proving that g is strictly increasing with rescpect to the first variable. 

The continuity (with respect to both the norms) of g, (jl7p and the fact that g is strictly 
increasing with respect to the first variable lead to have 

5||.||P(y) = 5||.||_,P(F) = {<7 = 0} = U (W}x{r?i}). 

2. We will intend the topological notions referred to || • ||. Firstly we prove that 

lim V{rj) = -00, Vf/ G dV{V). 

Let fj G dViy) and let {rj'^) C T^iV) be a sequence such that rj^ — > fj. We can suppose without 
loss of generality that (r/") C B^f], 1). Set 

rE"(.):=x(-;r7",0), := sup x^i^- 

Se[o,2T] 
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Thanks to Lemma 13.91 there exists K > such that < K for any n G N. So, since foix,y) < 
Co(l + + \y\) for some Cq > 0, we have for the dynamics of in the interval [0, 2T] 

<rx"(t) + i?, 

where 

R := Co (l + A'+ ||a||i2 (||f?i||i. + 1) + ||a||^. TI/^k) . 



Therefore there exists C > such that, for any s G [0,T), n G N, 

< x"(s) e''(*"^) + - (e''^*-'') - 1) < + C7(t - s)) + C7(t - s), t G [s, 2T1. (18) 

r 

By continuity of g we have hm„,_>oo 5(^0 ' "^"(O) = 0- Thus for any e > we can find no G N 
such that, for n > no, there exists s„ G [0,T) such that 

x'^isn) < e. (19) 

We want to show that 

r+00 

/ e-P*U2{x''{t))dt — ^-00, n^oo. (20) 

For this purpose, since U2 is bounded from above, it is clear that we can assume without loss of 
generality U2{-) < 0. We have for n > no, taking into account (jlSp and (jl9p . 

r.+cxD /■2T 



/ e-''*[/2(x"(t))dt < e-^P^ U2{x''{Sn){l + C{t-Sn)) + C{t-Sn))dt 
Jo Jsn 

/ U2{e{l+C{t-Sn))+C{t-Sn))dt 



r2T 



g-2pT /-CT 



Therefore, by the arbitrariness of e and since f/2 is not integrable at 0"*", we get ()20p . This is 
enough to conclude that J(?/"; 0) — > —00, as n ^ 00. Of course we have x"(-) > x(-; r?"", c(-)) for 
any c(-) G C(r/"). Since C/i is bounded from above this is enough to say that also V{rj'^) —00, 
as n — > cx). 

Now we prove the claim. Let f/ G dViV) and (r/") C T^{V) be such that r/" fj, suppose 
without loss of generality (rjn) C B{f], 1) and set x"'(-) := x{- ; (?7q + 1,7/"), 0) > 0. Since /o is 
Lipschitz continuous and nondecreasing on the second variable, there exists C > such that 

/o (^x{t),j\{0x{t + 0d^^ > -C (1 + x(t) + ||a||^2^(||77i|L2_^ + 1)) =: -R. 

Suppose ^ < . Then Jja;"(t) > rx''{t), so that, since rj'^ > 0, we have j;"(t) > r/[^ + 1 > 1. This 
leads to the estimate 

y(r?o" + !,<(•)) >i^, nGN, 
for some K > 0. Due to the concavity of V we have the estimate 

v,,{vn > F(7?o" + !,<(•)) - ^(%^<(•)) > ^ - - +00, 
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i.e. the claim. 

Suppose then ^ > and set := x(- ; (ryj + ^/r, r?]^), 0). Then > rx'^{t)-R, so that, 

since rjQ > 0, we have x^{t) > R/r > 0. This leads to the estimate 

V{v^ + R/r,7]^{-))>K, nGN, 

for some K > 0. Due to the concavity of V we have the estimate 

Vvoi^n > i \v{ri^ + R/r,7f,{-))-V{v^,vn-))\ > ^ [i^ - r?r(-))] - +oo, 



i.e. the claim. 



□ 



Proposition 3.11. Let (jl6p /loW, /ei rj € and consider the closed-loop delay state equation 
(jlip . r/ien t/iis equation admits a solution x*{-) G C^([0, +oo); M). Moreover, for all t>0, 



x*{t) > 0, {x*{t),x*{t + Okel^T,o]) e 

In particular the feedback strategy defined in (fT^ is admissible. 

Proof. Thanks to Lemma 13.101 if U2 is not integrable at we can extend the map C to a 
continuous map defined on the whole space {H, \\ ■ ||) defining C = on D{Vy. We set 

G{v) := rrio + /(??)- C (r/) , r, e H, 

and note that G is continuous. 

Local existence. Let f] H the initial datum for the equation. We have to show the local 
existence of a solution of 

fx'(t) = G((x(t),x(t + Ol€e[_T,o])), 
\x{0) = fjQ, x{s) = s G [-T,0), 

Since G is continuous, there exists 6 > such that m := sup||^_^||2<;, < +00. By continuity 

of translations in L^(R;M) we can find a G [0, T] such that 

/ '\m{t + 0-m{0\^dC<b/A, VtG[0,a]; 

J-T 

moreover, without loss of generality, we can suppose that Z^lviiOl^d^ < 6/16. Set 
Define 

M:={x(-)GC([0,a];]R) | - r?oP < 6/2} ; 
M is a convex closed subset of the Banach space C([0, a]; M) endowed with the sup-norm. Define 

x{t + 0-=m{t + 0, ift + ^<o. 
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and observe that, for t £ [0, a], x{-) G M, 

[' \x{t + C)-fj,{0\^d^ < [\2\x{t + 0\^ + 2\fii{0\^)d^ 
j-t J-t 



< 



y° (2 {\x{t + 0- Vo\? + 2|r?oP) + 1° Imm^dC 



2^2 + 1^-0^+1^ 



< b/4: 



< 2 

So, for t € [0, a], x{-) S M, we have 

||(x(t),x(t + Ol5e[-T,o]) -'?||' < \x{t)-mf' + J jx{t + 0-m{a''d^+ I \m{t + 0-vi{0\''dC 

< 6/2 + 6/4 + 6/4 = 6. 

Define, for t G [0,a], x(-) G M, 

[^x](i) :=f/o+ / G(x(s),x(s + C)l5G[-T,o])rfs, t€[0,a]. 

JO 



We have 



[Jx]{t)-m 



< 



G {x{s),x{s + ^)\^^[_Tfl])\ds 



< tm< 6/2. 



Therefore we have proved that J' maps the closed and convex set M in itself. We want to 
prove that J' admits a fixed point, i.e., by definition of J', the solution we are looking for. By 
Schauder's Theorem it is enough to prove that J" is completely continuous, i.e. that J[M) is 
compact. For any x{-) € M, we have the estimate 

[J x\{t) - [J x\{t) < \G{x{s),x{s + C)\^^i^Tfl])\ds <m\t-t\, t,te[0,a]. 

JtAt 

Therefore J{M) is a uniformly bounded and equicontinuous family in the space C([0,a];M). 
Thus, by Ascoli-Arzela Theorem, J{M) is compact. 

Global existence. Let r] G and let be the solution of equation (jlip defined on an 
interval [0,/3), /? > 0. Note that, by continuity of /o, C, we have x*{-) G C^([0, M). 
Since C(-) > 0, we have < x(-;?7,0); therefore x*(-) is dominated from above on [0,/3) by 

max x(-;rj, 0). 

te[o,(S] 

We want to show that it is also dominated from below in order to apply the extension argument. 
Let us suppose that x*{t) = for some t G [0,/3). We want to show that this leads to a 
contradiction, so that, without loss of generality, we can suppose that 

t = mm{t G (0,/3) I x*{t) = 0}. 

Therefore > in a left neighborhood of t. Since /o satisfies ([2]) and thanks to ()16p -(ii). we 
must have ■^x*{t) > 0, which contradicts x*{-) > in a left neighborhood of t. Therefore we 
can say that x*{-) > on [0, (3), so that in particular is bounded from below by on [0, /3). 
Therefore, arguing as in the classical extension theorems for ODE, we could show that we can 
extend x*{-) to a solution defined on [0, +00) and, again by the same argument above, it will be 
> on [0,+cx)). □ 
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4 Approximation results 



In this section we obtain some approximation results which may be used in order to produce 
e-optimal controls for a wider class of problems. Herein we assume that 

rx + fo{x,0) >0, Vx>0, (21) 

which implies in particular 

35 >0 such that rx + /o(x,0) > 0, Vx G [0,6]. (22) 

In fact, all the results given below hold under ()22p as well. We assume (I2ip only to simplify the 
proofs. Moreover we incorporate the term rx in the state equation within the term /o, so that 
consistently with ()2ip we assume that 

/o(x,0)>0, V2;>0. (23) 

Recall that, for a control problem, an e-optimal strategy is a strategy e-near to optimality. 
Precisely, in our problem 

Definition 4.1. Let rj € V(V), e > 0; an admissible control c^(-) G C{rj) is said e-optimal for 
the initial state r] if J{r]; c^(-)) > V{ri) — e. □ 

We will use the same concept for the control problems defined in the following. 



4.1 The case without utihty on the state 

In the previous section we introduced an assumption of no integrability of the utility function 
U2- This was necessary in order to ensure the existence of solutions for the closed loop equation 
and the admissibility of the feedback strategy. This fact is quite uncomfortable, because usually 
in consumption problems the objective functional is given by a utility depending only on the 
consumption variable, i.e. the case U2 = ^ should be considered. Of course we could take 
a U2 heavily negative in a right neighborhood of and equal to out of this neighborhood, 
considering this as a forcing on the state constraint (states too near to must be avoided). 
However we want to give here an approximation procedure to partly treat also the case C/2 = 0, 
giving a way to construct at least e-optimal strategies in this case. 
So, let us consider a sequence of real functions (C/2 ) such that 

C/2" T 0, C/2" not integrable at 0+, = on [1/n, +00). (24) 

Let us denote by J" and V"^ respectively the objective functionals and the value functions of 
the problems where the utility on the state is given by U2 and by and respectively 
the objective functional and the value function of the problem where the utility on the state 
disappears, i.e. C/2 = 0. It is immediate to see that monotonicity implies 

T 5 < V^. (25) 

Thanks to the previous section, for any problem y"', n G N, we have an optimal feedback 
strategy c* (-). 
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Lemma 4.2. Let rj G ViV^) C Then, for any e > 0, there exists an e-optimal strategy 

c^(-) € C(ry) for V^{ri) such that 

inf x{t;r],c%-)) > 0. 

tG[0,+oo) 

Proof. Let e > and take an e/2-optimal control c'^/^(-) G C(ry) for V'^{ri). Let M > T be 
such that 

-e-P^\Ui-UiiO)) <e/2. (26) 
P 

Define the control 

'c^/2(t), fortE[0,M], 



By Lemma 12.61 we have 



^0, fort>M. 

x{-,r],c'{-))>x{-,rj,c'/\-)) 
and, by the assumption (j23|) and since c'^(t) = for t > M, it is not difficult to see that 

x(t;r/,c^(-)) > 2;(M;?7,c^(-)), for t > M, 

so that 

inf x{t;r,,c'{-)) = inf x{t;r,,c'{-)) > 0. 
te[o,+oo) te[o,A/] 

We claim that c^(-) is e-optimal for V^{r]), which yields the claim. Since c^^^(-) is e/2-optimal 
for V^{rj), taking also into account (p6|) . we get 

y°(7?)- / e-P'Ui{c'{t))dt = V\7]) - e-P*Ui{c'''^{t))dt 
Jo Jo 

" + 00 



r+oo 

+ / e-f\Ui{c'/\t)) - UiiO))dt < e/2 + e/2 = e. 



□ 

Proposition 4.3. Let rj G 'D{V^) and e > 0. Then V"'{r]) — > V^{r]) and, when n is large enough, 
c*(-) is e-optimal for V^{r]). 

Proof. Let s > and take an e-optimal control c^(-) G C{^) for V^{r]) such that (Lemma 

KM 

m := inf x{t;r]; c"^ {■)) > 0. 

tG[0,+oo) 

Take n G N such that 1/n < m. Since C/^ = on [m, +oo), we have 

r+oo 

V\r]) - e < J{rj; c'{-)) = / e-P'Ui{c'{t))dt 

Jo 

r+oo 

e-'''[Ui{c^t)) + U^{x{t;v,c' {■)))] dt 

= r{ri,c^-)) < V^iv) = r{ri,cl{-)) < J\r,,cl{-)). 
The latter inequality, toghether with (j25p . proves both the claims. □ 



18 



4.2 The case with pointwise delay in the state equation 

In this subsection we want to show that our problem is a good approximation for growth models 
with time to build and concentrated lag and discuss why our approach cannot work directly 
when the delay is concentrated in a point. In this case the state equation is 



'y'{t) = fo{y{t),y{t-^))-c{t), 
y{0) = Vo, y{s)=rjiis), sG [-T,0). 



(27) 



It is possible to prove, as done in Theorem 12.41 that this equation admits, for every r] € H^, 
and for every c(-) € L;^^^([0, +oo); M), a unique absolutely continuous solution. We denote this 
solution by r], c(-)). The aim is to maximize, over the set 



Cadiri) ■■= {<■) G i.L([0,+oo);M) | y(-;r?,c(-)) > 0}, 



(28) 



the functional 



[UM-))+U2{y{t;r^,c{-mdt. 



Denote by Vq the associated value function. By monotonicity of /o we straightly get C 



Let us take a sequence {ak)kGN C W^'^ such that 

afc(-r) = 0, ||afc||i2^<l, (Il6|)-(ii) holds true Va;,, ^ 5_t/2 in {C{[-T,0];R))* , (29) 

where S_x/2 is the Dirac measure concentrated at —T/2. We denote by Xk{-;r],c{-)) the unique 
solution of dl]) where a(-) is replaced by afc(-). 

Proposition 4.4. Let rj G H^, c(-) G L;^^^([0, +oo); M) and set y{-) := ?/(•;??, c(-)), Xfc(-) := 
Xfc(-; ry, c(-)). Then there exists a continuous and increasing function h such that h{0) = and 



sup \xkis) - y{s)\ < h{t)uk(t), tG[0,+oo), 

s£[0,t] 

where Uk{t) as k —> oo, unifromly on bounded sets. 

Proof. Note that 

fO 



(30) 



|Q^fc|l(C{[-T,0];IE 



sup 

ll/l|oo=l 



< 



Mim < hkhi^ ■ T^/' < (31) 



Let t > 0; we have, for any C G [0, t], 

\xkiC)-yiC)\ 



< c 



J fo(^Xk{s), J ^ak{£,)xk{s + C)dCj - fo(^y{s),y (^s 

Xkis)-yis)\ds+ [ [ akiO{xk{s + - yis + 0)dC 

Jo J~T 

1^ l'^ak{Oy{s + m-y{s-^^ 



ds 
ds 



ds 



(32) 
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Call 



and set, for s € [0, t], 



gk(t) := sup \xk{s) - y{s)\ = sup |xfc(s) - y(s)|, 

sG[-T,t] s€[0,t] 



Uk{s) := Cfg 



dr. 



Note that, for every s G [0, t], the function [— T, 0] 9 ^ i— > Xk{s + ^) — y(s + is continuous, 
therefore thanks to (|3T]) we can write from (|32]) 



gk{s)ds + lifc(t) 



Jo 



Therefore, setting K := Cjq(1 + T"*^/^), we get by Gronwall's Lemma 

9k (t) < Uk {t) + Kte^^k {t) =■■ h{t)uk [t] . (33) 
Note that, since Uk ™ (^^(["2^; 0]; I^))*; we have the pointwise convergence 



T 



akiOvis + Od^ 



moreover 

rO 

T 



akiOvis + Od^ 



< 11^^11x2^ • II y(s + Olcehr,o] 11^2 < Crj^d-) < +00, VsG[0,t], 



where the last inequality follows from the fact that the function [0, t] —i- L'^rp, s ^ y(s+^)|^g[_'r^o] 
is continuous. Therefore we have by dominated convergence Uk{t) — > 0. By (I33p we get (|30p . □ 



For A; € N, € H+, let 

CL(^) := {<■) e LL([0,+oo);M) | Xk{-,r],c{-)) > 0}, 
Consider the problem of maximizing over C^^(r/) the functional 



(34) 



-P'[UMt)) + U2{xk{t;r],c{-mdt 



and denote by Vk the associated value function. Note that, since we have assumed (j23|) . straightly 
we get C 'D{Vk) for every k £ N. Thanks to the previous section we have a sequence of 

optimal feedback strategies for the sequence of problems (yk{i]))k£^, in the sense that we have 
a sequence (c^(-))fcGN of feedback controls such that c^(-) € ^^^(r]) for every fc € N and 



Mv;c*ki-)) = sup Jfe(7?;c(-)) =: Vkii]), 



\/k G N. 



Lemma 4.5. Zei ?] G 



• For any e > there exists an e-optimal strategy c^(-) G Cj^rf(r/) /or the problem Vo(r/) swc/i 



inf y{t;ri,c'{-))>0. 

tG[0,+oo) 
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• Assume that 

lim [xU2{x)] = — oo. (35) 

X— >0+ 

Then, for any e > there exists v > such that for any /c G N there exists an e-optimal 
control c|(-) G C^di^) f^'^ problem Vk{r]) such that 

inf Xk{t;ij,cU-))>u. (36) 

te[o,+oo) 

Proof, (i) Let e > and take an e/2-optimal control c^/^(-) E C^^(7/) for the problem Vo(r/). 
Take M > large enough to satisfy 

le-P^^{Ui-Ui{0)) <e/2. (37) 

Define the control 

^.(i) /^'^'W' fortG[0,M], 
^ [0, for t > M. 

A comparison criterion like the one proved in Lemma 12.61 can be proved also for equation (j27p . 
Therefore we have 

y{-,V,c'{-))>y{-,V,c'/'{-)) (38) 

and, since we have assumed (I23p . 



inf y{t;T],c%-)) = ini y(t;r],c%-)). 

iG[0,+oo) te[0,A/] 

We claim that y{-;r],(f{-)) is e-optimal for Vq, which yields the claim. Since c^/^(-) is e/2-optimal 
for Vq{i]), taking also into account (j37p and (j38p . 



hoo 



M)(r?)- / e-P' [Ulic'it)) + U2iyit;r^,c'i-mdt 



+ 00 



^o(^) - / e-''*([/i(c^/2(t)) + U2{y{t;r^,c'/\.)))dt 



+ / e"''\Ui{c'/\t))-Uimdt+ / e-^* C/2(y(t;7?,c^/2(.))_^^(y(^.^^^e(^))) 

< e/2 + e/2 = e. 

(ii) Due to ([25]) we have Xfc(-;?/,0) > ?7o for every A; S N. Let 

?7i(0) + C/2(7?o) 

Jo := ■ 

P 

Then we have Vk{r]) > Jfc(r?, 0) > jo for every /c E N. Take M > large enough to satisfy 

-e-P^^{Ui-Ui{0)) <e. (39) 
P 

Arguing as done to get (llSp and taking into account the comparison criterion, we can find 
Cm > such that, for every k £N and for every c(-) G C-'^d^v)^ we have for all s G [0, A/] and 
for all t G [s,M + 1], 

Xfc(t; 7?,c(-)) < Xfc(s; ??,c(-))(l + CAf (t - s)) + Cuit - s). (40) 
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Now take v > small enough to have 

(i) i.<l, (ii) ^<1, (iii) -^[/2(2^)e-^(*^+i)<jo-^^^^-l<0. (41) 

For /c € N, thanks to the previous section we have optimal strategies in feedback form c^(-) € 
C^^{ri) for Vk', we claim that Xk{t; t], c^(-)) > for t e [0, M] for every /c G N. Indeed suppose by 
contradiction that for some to ^ [0, M] we have Xk(to;r], cl{-)) = v; then by (jlO]) and (l^T]l -(i).(ii) 
we get that 

Xk{t]ri,cl{-)) <2v, forte " " 



2Cm 



Therefore, by ()4ip -(iii) 



j 1/ — — 

" e-P'U^{xk{t;ri,cU-)))dt < h - ^^^^ - 1. 
to P 

This shows that 

Jk{v,cU-)) <h-i<Vk{v)-i. 

This fact contradicts the optimality of c^(-)- Therefore we have proved that for the choice of 
given by (|^T]) we have 

Xk{t;r],cU-))>u, fortG[0,M]. 
We can continue the strategy c^(-) after M taking the null strategy, i.e. defining the strategy 

Then by ([23]) we have r?, c|(-)) > u for every /c e N. We claim that c|(-) is e-optimal for 
Vkiv) foi' every k £ N, which proves the claim. Indeed, taking into account the comparison 
criterion and (j39p for the inequality in the following, 

r+oo 

Vkir]) - / e-P\UMit)) + U2ixkit;v,cU-)))dt 
Jo 

r+oo 

= Vkiri)- / e-P\U,{cUt)) + U2{xk{t;v,cU-)))dt 
Jo 

r+OD r+oo 

+ / e-^*(C/i(4(t))-[/i(0))(it+ / e-''*([/2(xfc(t;r?,4(.))-f/2(xfc(t;r?,c|(t))))dt <e. 
Jm Jm 

□ 

Proposition 4.6. Let r] G H++ and suppose that ()35p holds true. We have Vfc(r/) — > Vo(r/), as 
k ^ oo. Moreover for every e > we can find a constant and a kg such that the strategy 
(c*f^ is the optimal feedback strategy for the problem ofVk) 



0, fort>Me. 



is e-optimal strategy for the problem Vo(7?). 
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Proof, (i) Here we show that 

\immfVk{v)>Vo{r,), (44) 

fc— >oo 

Let e > and let c^(-) G C°^(r/) be an e-optimal strategy for the problem Vo{r]). Thanks to 
Lemma 1131 - (i) we can suppose without loss of generality 2i/i := inffg[o^_|_oo) y{t; rj, c^{-)) > 0. The 
function Ui is uniformly continuous on [0, +oo) and the function U2 is uniformly continuous on 
[z^i,+oo). Let be a modulus of uniform continuity for both these functions. Take M > 
such that 

1 - - 1 — p-pM 1 

- -e~P''{Ui + U2) u;uM) + -e-P^Hum + U2{u,)) > -e, (45) 

P P P 

Define 

'c^(t), fortG[0,Af], 



4/W : = 
Let kM be such that 



0, for t > M. 







h{M)uk{M) < ui, yk > kM, (46) 

where Uk and h are the functions appearing in (j30p . Then, thanks to Proposition 14.41 and to the 
monotonicity property of /o, it is straightforward to see that Xk{t; rj, c%,j{-)) > z^i > 0, so that in 
particular c\j{-) E C^rf(^) for all k > kj^- For all k > kM, we have, thanks to Proposition 14.41 
and by definition of /cm, 

e-^'lUiicljit)) + U2{xk{t;v,ch{-mdt 

AI /■+00 

e-''[Ui{cl,{t)) + U2{xk{t;ij,cl,i-)))]dt + / e'^*[C/i(4,(t)) + f/2(xfc(t; 77, c|,(-)))]fit 

Jm 

> / e-P\Ui{c'{t)) + U2{y{t;v,c'{-)))]dt u^uA^i) + -e-^^^(C/i(0) + C/2(z.i)) 

Jo P P 

f+00 

> / e-f'[Ui{c^t)) + U2{y{t;r^,c^-)))]dt 
Jo 

1 - - 1 - p-P^ 1 

- -e-''^^(C7i + U2) LO,M) + -e-'''^(C/i(0) + U2{u^)). 

P P P 



so that by ([45]) 

^fc(??) > Vo{7j) - 2e, (47) 

which shows ()44p . 

(ii) Now we show that 

limsupVfc(r?) < Vo(??)- (48) 

fe^oo 

Let e > 0; thanks to Lemma H3] -(ii) we can construct a sequence {<ff^{-))ken, c|(-) € C^d{rj) for 
every A; € N, of e-optimal controls for the sequence of problems {Vk{r]))k£N such that 

2z^2 := inf inf Xfc(t; r?, 4(-)) > 0. 

fceNtG[0,+oo) 
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Let L0u2 be a modulus of uniform continuity for Ui on [0, +cxo) and for U2 on [z/2,+00). Take 
M > such that 

1 - - - 1 - p-P^ 1 

- -e-P^'{Ui + U2) uj,,{u2) + -e~P^'{Ui{0) + ^72(1^2)) > -e (49) 

P P P 



and define the controls 



1 0, for t > M. ^ ^ 



As before we can find fcj^-j such that we have 



h{M)uk{M) < 1/2, yk>kj^j, (51) 
In this case we have ?/(•;??, c|(-)) > 1^2 and 

e''*[f^i(cM/W) + f^2(y(t;r?,4^^,(-)))]dt 

M r+oo 

e-P'[Ui{cl^^{t)) + U2{yit;r^,cl^^{-)))]dt+ / e-^*[f/i(cL^(i)) + f^2(y(t; r?, ^ ^(•)))]cit 

' ' JM 

rM 1 _ -pM 1 

> / e-f'piiclit)) + i72(xfc(t;r?,4(-)))]dt ^.,(^^2) + -e-^^^(C/i(0) + [72(^^2)) 

Jo P P 

-00 



> / e-''*[C/i(4(t)) + [/2(xfc(t;r?,c|(-)))]dt 



1 - - - 1 — p~P^^ 1 

-e-P^\Ui + C/2) u^uM) + -e-^^'(^i(0) + [72(1^2)). (52) 

P P P 



By dMD, we get, for k > kj^, 



which proves (08]). 

(iii) The procedure of construction of c|, in (ii) yields e-optimal controls for the limit 
problem Vo{r]). Indeed, starting from e > 0, we can compute z/i,z/2,M, M depending on e such 
that (05]) and (jUJ hold true. Then, if {ak)k(m is chosen in a clever way, for example if (oA-OfceN is 
a sequence of gaussian densities, we can compute /cm, ^^7/ such that (|l6]) - ([5T]) hold true. Thanks 
to ([52]) and (07]), for every k > kM V kj^;^ the controls ^^(0 defined in ([50]) are 4e-optimal for 

the limit problem V^{r]). Replacing e with e/4 we get the controls in (03]). □ 

Remark 4.7. When the delay is concentrated in a point in a linear way, we could tempted to 
insert the delay term in the infinitesimal generator A and try to work as done in Section 12. 2i 
Unfortunately this is not possible. Indeed consider this simple case: 

(y'it) = ry{t) + y{t-T), 

\y{o) = m, y{s) = m{s), [-r,o). 

In this case we can define 

A:V{A)CH^H, {rjo,m{-)) ^ Ho + m{-T),v'ii-))- 
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where again 

V{A) := {?7 G if I r?i(-) G W''^{[-T,0];R), r?i(0) = r/o}. 
The inverse of A is the operator 

'Vo-c 



: {H, 



c + 



T 



where 



T 



r + 1 r + 1 

In this case we would have the first part of Lemma 13.91 but not the second part, because it 
is not possible to control |ryol by Indeed take for example r such that = i, and 

(?7")„gFj C H such that 



^0 



We would have c = 1/2, so that 



? = 1/2, r = 1, n G N. 

J-T 



0. Moreover we can choose r/" such that, when n oo, 



+ 



ds — > 0. 



Therefore we would have |77g | = 1/2 and — > 0. This shows that the second part of Lemma 
3.9l does not hold. Once this part does not hold, then everything in the following argument breaks 
down. ■ 



4.3 The case with pointwise delay in the state equation and without utihty 
on the state 

Now we want to approximate the problem of optimizing, for rj G ii++, 

r+oo 

Joir],ci-)) := / e~P'Uiicit))dt, 
Jo 

over the set (j28p . where y{-;r], c(-)) follows the dynamics given by ([27]). Let us denote by the 
corresponding value function and let us take a sequence of real functions {U2) as in (I24p . but 
with the assumption of no integrability at O"*" replaced by the stronger assumption 

lim X Unix) = -00, Vn G N. 

Fix n G N and consider the sequence of functions {atjk&n defined in ()29p . For A; G N, consider 
the problem of maximizing over the set C^^{ri) defined in i\3A\i the functional 

r+00 

Jj:iri,ci-)):= / e-'^\Ui{c{t)) + U^{xk{t;rj,ci-)))dt, 
Jo 

where Xk{-;r],c{-)) follows the dynamics given by ([I]) when a(-) is replaced by ak{-), and denote 
by the associated value function. 

Moreover, for k £N, consider the problem of maximizing 

r+00 
Jo 
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over C^^{r]) and denote by the associated value function. 

Finally consider the problem of maximizing over the set C^^iiv) the functional 

r+oo 

J^{rj,c{-)) := / e-P\UMt)) + U^{y{t;rj,c{-)))dt, 



and denote by the associated value function. 

For fixed n € N, the problems approximate, when k ^ oo, the problem Vg^ in the sense 
of Proposition 14.61 i-S- we are able to produce k^^n,^e,n large enough to make the strategy 
^ke.n,Me,n{') defined as in ([33|) admissible and e-optimal for the problem VQ"(r/). 

Proposition 4.8. Let tj € let kir^n,Me,n, Cks,„,Ms^„{-) as above. For every e > we can 

find He such that 

hmy,";^^(r?) = yo°(r/). (53) 

Moreover the controls c^^ ^ ,Men (") defined as in ()43p are admissible and 3e-optimal for the 
problem Vo'(r/). 

Proof. Let e > and consider the strategies c^{-) and c^f(-) defined as in the part (i) of 
the proof of Proposition 14. 6i Notice that actually M = M{e,n) =: M^. Notice also that by 
definition of /c^^n, we have Xk^^„{-;r],c%j^{-)) > vi and that ([37]) in particular implies 

-e-''*^"(C7i-C/i(0)) <e. 
P 

Take G N such that l/n^ < ui (notice that vi depends on e and does not depend on n). 
Then, since Ul^^ = on [z/i, +cxo), we can write 

f+oo 

VS{v)-e< 4{v;c%-)) = / e-^*[/i(c^(t))di 



e-f" 



U,ic^t)) + U^^{xk,^^^{t;v,c%i. (•))) 



dt 



so that 



limmfV,^;Jr^)>V,\v)- (55) 



On the other hand the strategies c^^ ^^m^ „(") defined in (|43|) are admissible for the problem 
Vq" (since the state equation related to V^" and to Vq is the same) and Cfc^„,Me„(") is e-optimal 
for V^" ^ (r]) for every n S N. Therefore 

v,^;jrj)-e< Jl:Jmck^,r.^M.,^S-)) < JI ,Jv; ck^,r.^,M.„J-)) = ^o(^;cfc,„„M,„,(-)) < v^'iv), 

(56) 

which shows 

limsupl^,'!Jr?)<yo°(r/). (57) 

Combining (155]) and (157]) we get ([53]) . Combining (15^ and (f56l) we get 

V^o°('?)<^o(^;cfc,„„Af,„,(-)) + 3e, 
i.e. the last claim. □ 
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